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Abstract 

Neural network models comprising elements which have exclusively excitatory or inhibitory synapses 
are capable of a wide range of dynamic behavior, including chaos. In this paper, a simple excitatory- 
inhibitory neural pair, which forms the building block of larger networks, is subjected to external 
stimulation. The response shows transition between various types of dynamics, depending upon the 
magnitude of the stimulus. Coupling such pairs over a local neighborhood in a two-dimensional 
plane, the resultant network can achieve a satisfactory segmentation of an image into "object" and 
"background". Results for synthetic and and "real-life" images are given. 



1 Introduction 



Dynamical transitions in brain activity, in the presence of an external stimulus, has received considerable 
attention recently. Most investigations of these phenomena have focussed on the phase synchronization 
of oscillatory activity in neural assemblies. An example is the detection of synchronization of "40 Hz" 
oscillations within and between visual areas and between cerebral hemispheres of cats |IJ] and other 
animals. Assemblies of neurons have been observed to form and separate depending on the stimulus. 
This has led to the speculation that, phase synchronization of oscillatory neural activity is the mechanism 
for "visual binding" - the process by which local stimulus features of an object (e.g. color, motion, 
shape) , after being processed in parallel by different (spatially separate) regions of the cortex are correctly 
integrated in higher brain areas, forming a coherent representation ("gestalt"). 

Sensory segmentation, the ability to pick out certain objects by segregating them from their surroundings, 
is a prime example of "binding" . The problem of segmentation of sensory input is of primary importance 
in several fields. In the case of visual perception, "object-background" discrimination is the most obvious 
form of such sensory segmentation: the object to be attended to, is segregated from the surrounding 
objects in the visual held. Several methods for segmentation, both classical B [| and connectionist (e.g., 
see Q) are reported in literature. 

Most of the studies on segmentation through neural assembly formation has concentrated on networks 
of oscillators which synchronize when representing the same object. Malsburg and coworkers p] have 
sought to explain segmentation through dynamic link architecture with synapses that rapidly switch 
their functional state. Similar approaches using synaptic couplings which change rapidly depending on 
the stimulus have been used in a neural model for segmentation by Sporns et al p]. Grossberg and 
Sommers have performed figure-ground separation with a network of oscillators, some of which belong 
to the "object" and the others to the "background" . Oscillations of the former are synchronized, whereas 
the others have non-oscillatory activity. Han et al || have used an oscillatory network for Hopheld-type 
auto-association in pattern segmentation, using the temporal dynamics of the nonlinear oscillators driven 
by noise and subthreshold periodic forcing. If the input is a superposition of several overlapping stored 
patterns, the network segments out each pattern successively, as synchronous activation of a group of 
'neurons'. Similar segmentation through synchronization of activity among a cluster of neurons have 
been shown by other groups || [n], O, [l2| . In contrast to this approach, we present a method of utilizing 
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the transition between different types of dynamics (e.g., between fixed-point and periodic behaviors) of 
the network elements, for performing segmentation tasks. 

In this paper, we investigate the dynamical response of an excitatory- inhibitory network model (with 
autonomous chaotic behavior) to external stimulation of constant intensity / (in time). We focus on how 
the behavior of an individual element within the network changes with /. A theoretical analysis has 
been presented for the transition from period-2 cycles to fixed-point behavior for an isolated excitatory- 
inhibitory pair (i.e., not coupled to any other element). Simulation results for the spatially interacting 
coupled network are presented. The application of the system for segmenting gray-level images is studied. 
Finally, the possible improvements of the proposed method are discussed. 



2 The Excitatory-Inhibitory Network Model 

The model, we have based our investigations, comprise excitatory and inhibitory neurons, coupled to 
each other over a local neighborhood. As we are updating the model only at discrete time intervals, it 
is the time-averaged activity of the neurons that is being considered. Therefore, the "neuronal activity" 
can be represented as a continuous function, saturating to some maximum value, inversely related to the 
absolute refractory period. This is arbitrarily taken to be 1, so that we can choose the sigmoid function 
to be the neural activation function for our model: 

Fn(z) = 1 - cxp(-/iz), if z > , . 

= 0, otherwise. 

The basic module of the proposed network is a pair of excitatory and inhibitory neurons coupled to each 
other (Fig. 1(a)). If a; and y be the activities of the excitatory and the inhibitory elements respectively, 
then they evolve in time according to: 

Vn+i = F b (w yx x n - w yy y n + I' n ), 

where, Wij is the weight of synaptic coupling from element j to element i, F is the activation function 
defined by (|l| and /, I' are external stimuli. By imposing the following restriction on the values of the 
synaptic weights: 

w xy _ w yy _ ^ 

and absorbing w xx and w vv within a and b (respectively), we can simplify the dynamics to that of the 
following one-dimensional difference equation or "map" : 

z n+1 = F a (z n + I n ) - kF b (z n + I' n ). (3) 

Without loss of generality, we can take k = 1. In the following account we will be considering only 
time-invariant external stimuli, so that, for our purposes: 

In =l' n =L 

The resultant neural map, exhibits a wide range of dynamics (fixed point, periodic and chaotic), despite 
the simplicity of the model [O, H . 



2.1 Dynamics of a single excitatory-inhibitory neural pair 



The autonomous behavior (i.e., = 0) of an isolated pair of excitatory-inhibitory neurons show a 
transition from fixed point to periodic behavior and chaos with the variation of the parameters a, b, 
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following the 'period-doubling' route, universal to all unimodal maps [p.5| . The map 

z n +i = F(z n ) = F a {z n ) - F b (z n ), (4) 

describing the dynamics of the pair (Fig. 1(b)), has two fixed points, at z\ — and z^ (which is the 
solution of the transcendental equation z — exp(-bz) — exp(— az)). The fixed point, z* is stable if the 
local slope ~ (a — b)) is less than 1. For 
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where fi = -, this condition no longer holds and z\ loses stability while z\ becomes stable, by a transcrit- 
ical bifurcation. On further increase of a (say), this fixed point also loses stability, with the local slope 
becoming less than -1, and a 2-period cycle occurs. Increasing a leads to a sequence of period-doublings 
ultimately giving rise to chaos. 

The introduction of an external stimulus of magnitude I has the effect of horizontally displacing the map, 
Eqn. ([!]), to the left by /. This implies that z\ — is no longer a fixed point, while the other fixed point 
z| is now a solution of the equation z = exp(—b(z + 1)) — cxp(— a(z + /)). The slope at z£ decreases with 
increasing / - giving rise to a reverse period-doubling transition from chaos to periodic cycles to finally, 
fixed-point behavior. 



2.2 Analysis of response to constant magnitude external stimulus 



We shall now consider how the dynamics of the excitatory-inhibitory pair changes in response to external 
stimulus. Let us consider the isolated neural pair, whose time evolution is given by Eqn. (^|) . On replacing 
the expression of the transfer function from (Q) , we get 

z n+ i = exp(-6(z„ + /)) - exp(-a(z„ + J)). (5) 

Now, 

* 

Zn+l — Z n Z 

for a fixed point. It is stable if 

dz n+ i > ^ 
dz n ~ 

i.e., 

(a - b) exp(-a(z„ + /)) - bz n+ i > -1. 

Therefore, for the fixed point to be marginally stable (i.e. = —1), it must satisfy the following 

condition: 

(a - b) exp(-a(z* + J c )) = bz* - 1, (6) 
where, I c is the critical external stimulus for which z* just attains stability. 

Let us define a new variable, a, as 

bz*-l 

a = -. (7) 

a — b 

Therefore, from (J|), we get 

exp(-a(z* + J c )) = a. (8) 

Also from (@), 

z = - H a (9) 

o /j, 
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„ := _(l +aa )l/M. (11) 



where fj, — b/a. Now, from (ph, a marginally stable fixed point can be expressed as 

z* = - exp(-a(z* + Z c )) + exp(-o(z* + J c )). 
Therefore, from (@) and ([)]), the above expression can be written as 

a" -a=\ + {^—^)a. (10) 
o /i 

By simple algebraic manipulation, we get 

1 

b 

Assuming aa << 1, we need to consider only the first order terms in a in the right hand side, so that 

&/i> a = l+™, (12) 
A 1 

which gives the following expression for a: 

a = w^- (13) 

For a real solution of z* to exist, we must have bz* — 1 > 0, since, otherwise, z* will have an imaginary 
component (from (|l6|)). In other words, a > (from (0)). Therefore, from (p"3|), we must have 

a < /ib 1 ^. (14) 

Since b — fia, we get 

a > . (15) 
For example, if /i = 0.5 then a > 8 for z* to be real. From (^) we get 

exp(a/ c ) = ^ b ^ exp(-az*). (16) 



Taking logarithms on both sides, we have, 



I c = -z* - - log(a). 
a 



Therefore replacing z* from 



I c = (£— L)a- J--ilog(a). (17) 
fxa a 

The equation (|l7]), together with (|l3|), provides the critical value of the external stimulus which leads the 
oscillatory neuron pair to a fixed stable state, subject to the restriction (|l5|). 



This expression can be further simplified. From (|ll|), we can write 

/xlog(a) = - log(6) + log(l + aa). 

As before, assuming aa << 1, we need to consider only the first order terms in a in the right hand side 
of the logarithmic expansion, which gives us 

log(a) = — --log(6). (18) 

[A [L 

From djj), (0), and (U), the critical magnitude of the external stimulus is given as 
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h= , " a +-l0g(^), 



(19) 



where e = exp(l). Fig. 2 shows the a vs. I c curves for different values of /x, viz. fi = 0.1, 0.25 and 0.5. 



4 



2.3 Choosing the operating region 



To make the network segment regions of different intensities (ii < I2, say), one can fix fx and choose a 
suitable a, such that I\ < I c < 1%. So elements, which receive input of intensity I\, will undergo oscillatory 
behavior, while elements receiving input of intensity I2, will go to a fixed-point solution. Notice that, the 
curves obtained from gives two values of a for the same I c . This gives rise to an operational question: 
given a certain I c , which value of a is more appropriate? Notice that, the region of the a vs I c curve 
(Fig. (2)) to the left of the maxima, has a very high gradient. This implies that, in the presence of wide 
variation in the possible value of I c , choice of a from this region will show very small variation - i.e., the 
system performance will be robust with respect to uncertainty in the determination of the appropriate 
value of I c . This is possible in the case of any gray-level image with a bimodal intensity distribution, 
having a long, almost uniform valley in between the two maxima. 

On the other hand, the region of the curve to the right of the maxima has a very low gradient (almost 
approaching zero for high values of a) . This implies structural stability in network performance, as wide 
variation in choice of a will give almost identical results. So, choice of a from this region is going to make 
the network performance stable against parametric variations. As both robustness against uncertain 
input and stability against parametric variations are highly desirable properties in network computation, 
a trade-off seems to be involved here. The nature of the task in hand is going to be the determining 
factor of which value of a we should choose for a specific I c . 



3 The two-dimensional network of excitatory-inhibitory neural 
pairs 

The introduction of spatial interactions over a local neighborhood in the above model produces some 
qualitative changes in the response of the network to external stimulus. We have considered discrete 
approximations of circular neighborhoods [jl6[ of radii r ex ,ri n (r = 1,2) in our simulations (Fig. 3(a)). 
A theoretical study of the changes in the dynamics due to spatial interactions is in progress and will be 
reported later. 

There is an important feature to consider about the relative sizes of the neighborhoods of the excitatory 
and inhibitory neurons, lZ ex and lZi n , respectively. The autonomous dynamics of a coupled network is 
given by 

Z n +1 — F(z„ + — J SieTCe^n — ,^ i ^ieTlinVn), 

where \TZ\ indicates the number of neurons within a neighborhood. This can be rewritten as 

z n+ i = F(z n + | Ste7Z, ea Zn + ^ ^ ien^Vn ~ j- s *erc„,2A)- 
If we take r ex = rj„ , then the equation reduces to 

z n+1 = F(z n + — — Eiew^X). 

I l^ex 

As z n > at all sites i, the activation increases with n, thereby driving the network to a homogeneous, 
uniformly activated state. 

Therefore, unless r ex < r,- n , the network activity becomes unstable owing to the unbounded increase in 
the activity of the excitatory elements. This is seen by looking at the averaged activity of the network, 
= jj'^iLi z n('i)i where z(i) indicates the i-th excitatory-inhibitory neural pair. For r ex — r in , 1 shows 
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an oscillatory behavior whose amplitude increases with n (Fig. 3(b)). For stable behavior, the amplitude 
of oscillation should be constant in time (this is so for r ex < rj n , as shown in Fig. 3(c)). 



4 Simulation and Results 

The response behavior of the excitatory-inhibitory neural pair, with local couplings, has been utilized in 
segmenting images and the results are shown in Figs. 4 and 5. Both synthetic and "real-life" gray-level 
images have been used. The initial state of the network is taken to be totally random. The image to be 
segmented is presented as external input to the network, which undergoes 200 - 300 iterations. Keeping 
a fixed, a suitable value of /i is chosen from a consideration of the histogram of the intensity distribution 
of the image. This allows the choice of a value for the critical intensity (J c ), such that, the neurons 
corresponding to the 'object' converge to fixed-point behavior, while those belonging to the 'background' 
undergo period-2 cycles. In practice, after the termination of the specified number of iterations, the 
neurons which remain unchanged over successive iterations (within a tolerance value, th) are labeled as 
the "object" , the remaining being labeled the "background" . 

The synthetic image chosen is that of a square of intensity I2 (the "object") against a background of 
intensity I\ [I\ < I2). Uniform noise of intensity e is added to this image. The signal-to- noise ratio 
(SNR) is defined as the ratio of the range of gray levels in the original image to the range of noise added 
(given by e). For SNR = 1, the results of segmentation are shown in Fig. 4. Fig. 4(a) shows the original 
image while segmentation performance of the uncoupled network is presented in Fig. 4 (b) . As is clear 
from the figure, the isolated neurons perform poorly in identifying the 'background' in the presence of 
noise. The segmentation performance improves remarkably when spatial interactions are included in the 
model. Results for r ex = l,rj„ = 2 and r ex = ri n = 2 arc shown in Figs. 4(c) and (d), respectively. The 
two architectures show very similar segmentation results, at least upto the iterations considered here, 
although the latter is unstable (as discussed in the previous section). Excepting for the boundary of the 
'object', which is somewhat broken, the rest of the image has been assigned to the two different classes 
quite accurately. 

We have also considered the 5-bit gray level "Lincoln" image (Fig. 5(a)) as an example of a "real-life" 
picture. A suitable I c has been estimated by looking at the histogram of the gray-level values, and 
taking the trough between two dominating peaks as the required value. As in the synthetic image, the 
performance of a network of uncoupled neurons is not satisfactory (Fig. 5(b)). The results of including 
spatial interaction are shown in Figs. 5(c) and (d). Most of the image has been labeled accurately, except 
for a few regions (e.g., near the neck). 

Note that, we have considered a single value of a (and hence I c ) for the entire image. This is akin to "global 
thresholding" . By implementing local thresholding and choice of a on the basis of local neighborhood 
information, the performance of the network can be improved. 



5 Discussion 

In the present work, we have assumed constant connection weights over a local neighborhood. However, 
a gaussian profile of weights may be biologically more realistic. One can also make the critical intensity 
I c proportional to the ambient intensity. This is in tune with how the retina seems to alter its sensitivity 
to incoming visual stimuli Finally, the role of variable connection weights, that can be implemented 
in the present model by changing the value of k (ratio of the weights), may be investigated. 
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Chaotic neurodynamics has been used to segment images by Hasegawa et al (18|. However, their method 
is based on using chaos to avoid the local minima problem in the variable shape block segmentation 
method. We have instead concentrated on using stimulus induced transitions in neural network dynamics 
to segment images. This is closer to the neurobiological reality. As Malsburg M has indicated, the reason 
oscillatory synchronization has been studied so far, as a mean of segmenting sensory stimuli is its relative 
ease of analysis. However, with the developments in nonlinear dynamics and chaos theory, we can advance 
to segmentation using more general dynamical behavior. 
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Figure Captions 

Fig. 1 (a) The basic excitatory (a;)-inhibitory (y) neural module, and (b) the 1-dimensional neural map, 
F (Eqn. (Q)), with the activation functions for the constituent excitatory (slope, a = 20) and inhibitory 
(b = 5) neurons. 

Fig. 2 Critical magnitude (7 C ) of the external stimulus, at which transition from periodic to fixed point 
behavior occurs. The circles (filled and blank) and squares represent the values obtained exactly through 
numerical procedures for b/a = /i = 0.5, 0.25 and 0.1, respectively. The curves indicate the theoretically 
predicted values. 

Fig. 3 (a) Discrete circular neighborhoods of radii, r = 1 and r = 2. Average activity (z) of a network 
of 100 x 100 elements, arranged in a two-dimensional plane, with coupling over a local neighborhood: (b) 
r ex — 2, Ti n — 2 and (c) r ex = 1, n n = 2. 

Fig. 4 Results of implementing the proposed segmentation method on noisy synthetic image: (a) original 
image, (b) output by the uncoupled network, (c) output by the coupled network (r ex = l,r,- n = 2), and 
(d) output by the coupled network (r ex — ri n = 2), after 200 iterations (a=20, 6/a=0.25 and threshold 
th=0.02). 

Fig. 5 Results of implementing the proposed segmentation method on "Lincoln" image: (a) original 
image, (b) output by the uncoupled network, (c) output by the coupled network (r ex — l,rj„ = 2), and 
(d) output by the coupled network (r ex = ri„ = 2), after 300 iterations (a=30, 6/a=0.25 and threshold 

th=om). 
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